A Recursive Partitioning Decision Rule for Nonparametric Lassification *

A new criterion for driving a recursive partitioning decision rule for nonparametric classification is presented. The criterion is both conceptually and computationally simple, and can be shown to have strong statistical merit. The resulting decision rule is asymptotically Bayes risk efficient. The notion of adaptively generated features is introduced and methods are presented for dealing with missing features in both training and test vectors. (Submitted to IEEE Transactions on Computers) *This work supported by U.S. ERDA under contract AT(043)515 Introduction In many classification problems, the underlying class conditional probability densities are either partially or completely unknown. Consequently, the classification logic must be designed from information measured from representative samples drawn from each class. The nonparametric classification problem may be stated in the following manner. A random p-dimensional vector of observed features, x", is thought to belong to one of M populations, K~, f12...flM, characterized by density distributions that are unspecified. On the basis of these features, a de-. cision is made as to which distribution function characterizes x", using a training set of vectors drawn from each of the populations, 7c1, 7~2."~~. The nonparametric decision rules that have received the most attention are the k-nearest neighbor decision rules first introduced by Fix and Hodges [1,2]. The training samples from the M populations are combined into a single population with each vector tagged as to the class from which it originated. The k closest training vectors to ?(with respect to a specified distance function and metric) are located,and x"is assigned to the class with the largest representation in this set. These authors investigated the rule for k +oo and showed that the procedure is asymptotically Bayes risk efficient, if k is chosen to be a function of the training sample size, N, such that lim k(N) = co, while lim[k(N)/N] = 0. N+CD N-+CC The rule for fixed k has been investigated by Cover and Hart [3]. They show that for the extreme case of k=l (nearest neighbor decision rule), the asymptotic probability of misclassification is bounded from above by P[2 W/(M-l)] where R* is the Bayes probability of misclassification.